Skip to content

Configure JFrog Artifactory as PyPI proxy for hardened runners#1379

Merged
hectorcast-db merged 12 commits intomainfrom
mihaimitrea-db/jfrog-artifactory-migration
Apr 7, 2026
Merged

Configure JFrog Artifactory as PyPI proxy for hardened runners#1379
hectorcast-db merged 12 commits intomainfrom
mihaimitrea-db/jfrog-artifactory-migration

Conversation

@mihaimitrea-db
Copy link
Copy Markdown
Contributor

@mihaimitrea-db mihaimitrea-db commented Apr 1, 2026

Summary

Hardened GitHub Actions runners block direct access to public registries (PyPI, files.pythonhosted.org). This adds OIDC-based JFrog Artifactory authentication so CI can download Python packages through the db-pypi proxy.

Uses the same jfrog/setup-jfrog-cli action as CLI PR #4875.

What changed

  • New composite action (.github/actions/setup-jfrog-pypi/action.yml): Uses jfrog/setup-jfrog-cli for OIDC authentication (same action + SHA as the CLI repo), then sets UV_INDEX_URL using the token from the action's oidc-token/oidc-user outputs.
  • test.yml: Added id-token: write permissions, JFrog setup step, and uv lock before make dev test.
  • push.yml: Added workflow-level id-token: write permissions (needed for workflow_call to test.yml). Added JFrog setup + uv lock to fmt and check-manifest jobs.
  • Makefile: Added fix-lockfile target. All existing targets are unchanged.

How it works

uv.lock stores full registry URLs per package. Setting UV_INDEX_URL to JFrog causes uv sync --locked to fail (stale lockfile). To work around this, CI runs uv lock after JFrog setup, which rewrites the lockfile with JFrog URLs while keeping the same versions. Then make dev test / make dev fmt work unchanged with --locked. The lockfile rewrite is ephemeral (never committed).

In the fmt job, git checkout -- '*.lock' restores the committed lockfile after formatting, then make fix-lockfile normalizes any accidentally committed proxy URLs. git diff --exit-code catches both unformatted code and dirty lockfiles.

make fix-lockfile

Replaces JFrog proxy URLs with their public PyPI equivalents in all *.lock files. Prevents proxy URLs from being accidentally committed to the repo.

When to use it: If you run uv lock while UV_INDEX_URL points to JFrog (e.g. after sourcing CI env vars), run make fix-lockfile before committing to normalize URLs back to public PyPI.

CI enforcement: The fmt job runs this before git diff --exit-code, so committed proxy URLs fail CI.

Out of scope

  • tagging.yml: Generated from the Universe codegen template (openapi/genkit/sync/workflows/tagging.yml). JFrog setup needs to be upstreamed there.
  • release.yml / release-test.yml: Need special publish runners per the migration guide. Separate follow-up with the security team.
  • Runner migration: Switching to hardened runner labels can be done as a follow-up.
  • Trigger Tests CI failure: Pre-existing IP allowlist issue (DECO-26816).

NO_CHANGELOG=true

Hardened GitHub Actions runners block direct access to public registries.
This adds OIDC-based authentication to JFrog Artifactory and routes all
uv package downloads through the db-pypi proxy. A `make fix-lockfile`
target prevents proxy URLs from leaking into the committed lockfile.
The JFrog proxy URL structure has /packages/packages/ (the proxy path
plus the original PyPI path), so the sed replacement must map
db-pypi/packages to files.pythonhosted.org (without /packages suffix).

Also, uv lock through JFrog drops size/upload-time metadata from the
lockfile. Restore committed lockfiles with git checkout before the diff
check so only formatting changes and accidentally committed proxy URLs
are caught.
The tagging workflow used setup-uv v4.2.0 without a version pin,
installing whatever uv release was latest. Align it with the rest
of the workflows: setup-uv v7.3.0 pinned to uv 0.6.5.
tagging.yml is generated from the Universe codegen template and is not
in the critical path for unblocking unit tests. The uv version pin and
JFrog setup need to be upstreamed to the genkit sync template instead.
Release workflows need special publish runners and coordination with the
security team per the migration guide. Keep this PR focused on unblocking
CI (tests, fmt, check-manifest).
Remove uv lock from CI and use uv sync --frozen instead of --locked.
If the hardened runners transparently route files.pythonhosted.org
through the JFrog proxy, downloads from lockfile URLs should work
without rewriting them.
uv run without --frozen triggers auto-sync, which re-locks the lockfile
with JFrog URLs and drops size/upload-time metadata. Pass --frozen to
all uv run calls since we already synced the environment explicitly.
Add UV_SYNC_FLAGS and UV_RUN_FLAGS variables to the Makefile so CI can
pass --frozen without duplicating commands. Local dev defaults remain
unchanged (--locked for sync, no flags for run).
Set UV_FROZEN=1 in the composite action so uv natively skips lockfile
re-resolution on all commands. This lets workflows call make dev test
and make dev fmt unchanged -- no Makefile modifications needed beyond
the new fix-lockfile target.
Replace the manual OIDC token exchange (~20 lines of curl/jq) with the
jfrog/setup-jfrog-cli action, which handles OIDC internally and exposes
the token via step outputs. Same action + SHA as the CLI repo (PR #4875).
uv rejects --locked and --frozen together. Add UV_LOCK_FLAG variable
(defaults to --locked) that the composite action clears via env var,
letting UV_FROZEN=1 handle lockfile behavior in CI instead.
Run uv lock after JFrog setup to rewrite the lockfile with JFrog URLs
(same versions, different registry). This lets make dev test and make
dev fmt work unchanged with --locked. In fmt, restore committed
lockfiles before the diff check so only formatting issues and
accidentally committed proxy URLs are caught.
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 7, 2026

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-py

Inputs:

  • PR number: 1379
  • Commit SHA: 5381c9da913bf07cc5379094f3bb9476090d0f63

Checks will be approved automatically on success.

@hectorcast-db hectorcast-db merged commit b16b649 into main Apr 7, 2026
12 of 13 checks passed
@hectorcast-db hectorcast-db deleted the mihaimitrea-db/jfrog-artifactory-migration branch April 7, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants